Method for perception measurement

ABSTRACT

In a procedure for measuring perception, first visual coordinates of a first point of vision assigned to a first visual field are processed, and second visual coordinates of a second point of vision assigned to a second visual field image, are processed. For determining the visual attention to certain areas of the surroundings, the second visual coordinates are examined together with the first visual coordinates in a comparison device to check whether they fulfill a fixation criterion. If the first and second points of vision fulfill the fixation criterion, they are assigned to a first fixation associated with ordered perception. Otherwise, they are assigned to a first saccade associated with aleatoric perception.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application is the U.S. National Stage of International ApplicationNo. PCT/AT2008/000210, filed Jun. 12, 2008, which designated the UnitedStates and has been published as International Publication No. WO2008/151346 and which claims the priority of Austrian PatentApplication, Serial No. A 911/2007, filed Jun. 12, 2007, pursuant to 35U.S.C. 119(a)-(d).

BACKGROUND OF THE INVENTION

The invention relates to a method for perception measurement, inparticular for measuring the visual attention of an individual.

An eye tracking system can be used to determine the area or location onthe visual field on which the individual's point of vision lies. Thistype of eye tracking system determines visual coordinates, in otherwords coordinates within the visual field of the individual, on whichthe individual's point of vision is focussed. An extremely exceptionaland precise procedure to determine these visual coordinates isidentified from EP 1 300 018 B1.

The purpose of the invention is therefore a method for perceptionmeasurement, in particular for measuring the visual attention of anindividual of the type described in the introduction, wherein the visualattention for certain areas can be measured as accurately as possible.

SUMMARY OF THE INVENTION

In accordance with the invention, this is achieved by a method formeasuring visual perception, having the steps of

processing at least first visual coordinates of a first point of visionassigned to a first field-of-view image and determined, for example byusing an eye tracking system, processing at least second visualcoordinates of a second point of vision assigned to a secondfield-of-view image, with the second field-of-view image being recordedafter the first field-of-view image, examining the second visualcoordinates of the second point of vision together with the first visualcoordinates of the first point of vision in a comparison device andchecking whether they fulfill at least one predetermined first fixationcriterion, assigning the first and second points of vision, providedthey fulfill the at least one first fixation criterion, to a firstfixation assigned to an ordered perception, and marking the first andsecond points of vision as such, and assigning the first and secondpoints of vision, if they do not fulfill the at least one first fixationcriterion, to a first saccade, to be assigned to aleatoric perception,and marking the first and second points of vision as such.

The perception of the test subjects or their attention on certainsurrounding areas can therefore be measured on a scientific basis. Byusing predefined surroundings, it is possible to determine exactly theareas which are perceived reliably and consciously by the test subjectsand the areas that are given a subordinate and secondary glance. Thisenables the quality of surroundings, such as a workplace, to be assessedand measured, particularly in safety-related or hazardous areas, forexample a road, particularly on bends, construction sites and/orthoroughfares, a user screen interface, switchboards, a machine controlpanel, the cockpit design of motor vehicles and aircraft, and/or anadvertising medium such as an image or text display or televisioncommercials. As a result, areas in the surroundings and the environment,which endanger life and limb, are assessed and measured on the basis oftheir degree of perceptibility, and reshaped in order to improve therecording of important information. The run of a road can be optimisedduring planning with regards to lowering the risk of accidents,important traffic signs, such as stop signs, can be positioned atspecific locations where road users demonstrate a high degree ofperception based on scientific research. Work surroundings can bespecifically designed so that important and safety-related controlelements, notices and operating elements encourage ordered perception bythe user; advertising can be adjusted to the ordered perception of theobserver.

The subclaims, which form part of the description along with Claim 1,concern additional beneficial embodiments of the invention.

BRIEF DESCRIPTION OF THE DRAWING

The invention is described in greater detail with reference to theenclosed drawings, which only present the preferred embodiments by wayof example. This shows:

FIG. 1 a block diagram of the first embodiment of the invention;

FIG. 2 a human eye in cross-section;

FIG. 3 a block diagram of a second embodiment of the invention;

FIG. 4 a block diagram of a third embodiment of the invention;

FIG. 5 a block diagram of a fourth embodiment of the invention;

FIG. 6 a schematic representation of eye glance behaviour for fixation;

FIG. 7 a schematic representation of eye glance behaviour with asequence for initial fixation, a saccade and a second fixation;

FIG. 8 a preferred embodiment of an output of the first relativedistance;

FIG. 9 a first preferred output of a visual field video with a first anda second circuit;

FIG. 10 a second preferred output of a visual field video with a thirdcircuit;

FIG. 11 a third preferred output of a visual field video with a fourthcircuit;

FIG. 12 a preferred user interface for a preferred computer-implementedembodiment of the invention;

FIG. 13 a first preferred output of the frequency of the fixationsdetermined depending on the angle of fixation;

FIG. 14 a first preferred output of the frequency of the saccadesdetermined depending on the angle of saccade;

FIG. 15 a first preferred output of the frequency of the fixationsdepending on the variable fixation criterion as a set of curves withconstant initial duration;

FIG. 16 a fourth preferred output of a visual field video;

FIG. 17 a fifth preferred output of a visual field video;

FIG. 18 a sixth preferred output of a visual field video;

FIG. 19 a seventh preferred output of a visual field video;

FIG. 20 an eight preferred output of a visual field video;

FIG. 21 a schematic representation of the part of an eye tracking systemconnected to the head of a test subject;

FIG. 22 a schematic representation of the eye diagram;

FIG. 23 a schematic representation of a visual field diagram;

FIG. 24 a ninth preferred output of a visual field video;

FIG. 25 a tenth preferred output of a visual field video;

FIG. 26 an eleventh preferred output of a visual field video;

FIG. 27 a twelfth preferred output of a visual field video;

FIG. 28 a preferred output layout in an initial view;

FIG. 29 a preferred output layout in a second view;

FIG. 30 a first preferred output of the frequency of the fixationsdetermined depending on the fixation duration;

FIG. 31 a first preferred output of the frequency of the saccadesdetermined depending on the length of saccade;

FIG. 32 a first preferred output of the frequency of the blinks counteddepending on the length of blinking;

FIG. 33 and FIG. 34 a first example of a preferred output layout for apreferred analysis tool; and

FIG. 35 and FIG. 36 a second example of the preferred output layout forthe preferred analysis tool.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

FIGS. 1, 3, 4 and 5 each show block diagrams of preferred embodiments ofa method for perception measurement, in particular for measuring thevisual attention of an individual, wherein at least first visualcoordinates of a first point of vision (37), associated to a first imageof a visual field, and at least second visual coordinates of a secondpoint of vision (38), associated to a second image of a visual field,are processed, first and second visual coordinates being detectedessentially by an eye tracking system. The invention is characterised inthat the second image of a visual field is recorded after the firstimage of a visual field and that the second visual coordinates of thesecond point of vision (38) are analysed with the first visualcoordinates of the first point of vision (37) in a comparative devicefor meeting at least one first predetermined fixation criterion (25).When the first fixation criterion (25) is fulfilled, the first andsecond points of vision (37, 38) are allocated to a first fixation (48)that can be associated to ordered perception, and when the firstfixation criterion (25) is not fulfilled, the first and second points ofvision (37, 38) are marked and allocated to a first saccade associatedwith aleatoric perception.

The perception of the test subjects or their attention on certainsurrounding areas can therefore be measured on a scientific basis. Byusing predefined surroundings, it is possible to determine the areaswhich are perceived reliably and consciously by the test subjects andthe areas that are given a subordinate and secondary glance. Thisenables the quality of surroundings, such as a workplace, to be assessedand measured, particularly in safety-related or hazardous areas, forexample a road, particularly on bends, construction sites and/orthoroughfares, a user screen interface, switchboards, a machine controlpanel, the cockpit design of motor vehicles and aircraft, and/or anadvertising medium such as an image or text display or televisioncommercials. As a result, areas in the surroundings and the environment,which endanger life and limb, are assessed and measured on the basis oftheir degree of perceptibility, and reshaped in order to improve therecording of important information. The run of a road can be optimisedduring planning with regards to lowering the risk of accidents,important traffic signs, such as stop signs, can be positioned atspecific locations where road users demonstrate a high degree ofperception based on scientific research. Work surroundings can bespecifically designed so that important and safety-related controlelements, notices and operating elements encourage ordered perception bythe user; advertising can be adjusted to the ordered perception of theobserver.

The concepts chosen in the current embodiment with regards to the first,second, third, etc. visual coordinates, points of vision, fixations,saccades, circuits, angle of fixation and/or saccade and so on are in noway preferred so as to restrict the method sequence to just two of theindicated features or to understand a single method sequence, butinstead as a description of the individual sequence of a predeterminedand often repeatable method.

The method according to the invention processes data, which is recordedusing a so-called eye tracking system. Such an eye tracking system isrepresented schematically in FIG. 21. A highly exceptional eye trackingsystem is described in EP 1 300 108 A1. Such an eye tracking system,which is described briefly below, works according to a method forrecording, evaluating and analysing glance sequences of a test subjectusing an eye tracking system, wherein the visual field of the testsubject is recorded using a first camera (76) rigidly connected to thehead (80) of the test subject so that it faces forward and is recordedin a visual field video, the movement of the pupils of the test subjectis recorded with a second camera (77), which is also rigidly connectedto the head (80) of the test subject, and is recorded in an eye video,and the eye video and the visual field video (9) are recorded on a videosystem and time-synchronised, wherein for each individual image of theeye video, therefore for each eye image (78) the pupil coordinates xa,yaare determined, the correlation function K between pupil coordinatesxa,ya on the eye video and coordinates xb,yb of the corresponding pointof vision B, i.e. the point the test subject fixes on, on which thevisual field image (79) of the visual field video (9) is determined, andafter determining the correlation function K for each individual imagefrom the pupil coordinates xa,ya on the eye video, the coordinates xb,ybof the corresponding point of vision B on the visual field video areextrapolated, wherein to determine the pupil coordinates xa,ya for eachindividual image of the eye video with a visual detection programme, thecontrasts of the pupils to the surroundings are automatically recorded,all points of the individual image, which are darker than a predefineddegree of darkness, are identified, these points record and limit anarea of darkness corresponding to the pupil and the focus of the area ofdarkness, which corresponds to the middle of the pupil with the pupilcoordinates xa,ya, is determined. If preferred it can be defined that apredetermined number of points on the edge of the pupil are selected,which can be easily and reliably identified due to their contrast withthe surroundings, and that these points are accepted as part of anellipsis, wherein the focus or centre of an ellipsis is calculatedaround the circumference of which the predetermined number of pointslie. This achieves a particularly high level of accuracy, which is farbeyond the state of the art of known eye tracking systems. Consequentlyerrors, which can be caused by reflections on the eye, have no influenceon the measured result. FIG. 22 shows a schematic example for an eyeimage (78) of an eye video with the pupil coordinates xa,ya. FIG. 23shows a schematic example for a visual field image (79) with thecoordinates xb,yb of the first point of vision (37). The purpose of theeye tracking system is to represent with the greatest possible accuracythe point on which the visual field of an individual fixes, i.e. theexact point to which the test subject's interest or attention is drawn.

The visual field is recorded by a first forward-facing camera (76)rigidly connected to the head (80) of the test subject. The movement ofthe pupils of the test subject is recorded by a second camera (77),which is also rigidly connected to the head (80). Rigidly connected inthis context means that both cameras (76, 77) are attached to the head(80) of the test subject in such a way that they move with the testsubject and follow all of the movements made by the test subject withoutrestricting the freedom of movement of the subject's head and eyes inany way. By evaluating these two recordings it is possible to ascertainwith considerable accuracy the point on which the test subject is fixed.Statements can also be made regarding visual attention, visualconnections and visual absences.

Such eye tracking systems are preferred for use in the safety sector,particularly in the area of accident research, as well as in the area ofadvertising, sport or other human physiological tests.

Overall, research into eye glance behaviour represents a significantbuilding block in the research of physiological causes of accidents. Forexample, comprehensive visual tests can ascertain new findings toexplain and reconstruct accidents in terms of human performance limits.

As a result, particularly hazardous points in road traffic can beinvestigated with the eye tracking system. A test subject fitted withsuch an eye tracking system travels through the hazardous point and eyeglance behaviour is recorded. The sum of the glances analysed isreferred to below as a glance sequence. By analysing eye glancebehaviour it is possible to ascertain which signposts or traffic signsare ignored because of their unfavourable positioning or where at ajunction there are points that are particularly ignored. In the area ofoccupational safety, e.g. on construction sites, tests can determinewhich hazards are perceived late by the test subject and which safetyprecautions would be required. A further important area of applicationfor eye tracking systems is the analysis of advertisements or televisioncommercials. In this area it is also possible to determine withconsiderably accuracy which messages, text, logos, etc. are perceived bythe test subject, for how long and in what order.

FIG. 21 shows part of an eye tracking system for the performance of apreferred method for determining the point of vision or visualcoordinates (VCO). The visual field of the test subject is recorded by afirst forward-facing camera (76) rigidly connected to the head (80) ofthe test subject. This first camera (76) therefore gives an approximateimage of the test subject's line of vision, which is defined purely bythe position of the head (80). The first camera (76) can be a CCD colourcamera, for example, which records the majority of the test subject'svisual field.

If preferred the first camera (76) and/or the second camera (77) canalso be controlled using software and thus be adapted to the externalconditions of use. This ensures that by directly recording the pupilsthere is no distortion in the pupil image and through the directproximity to the eye (33) a large image is generated and the set-up canbe kept smaller overall. Current methods represent a considerable sourceof inaccuracy due to their size and generally poor assignment of thepupil point. This results not only in difficulties in the weight of theeye tracking system, but also general restrictions on the eye glancebehaviour of the test subject, which are avoided by the method in thepresent invention. As a result, the eye tracking system in the presentinvention can also be used by test subjects with different clothing andprotective measures, such as a helmet, without restriction. It istherefore possible to use different lightweight cameras (76, 77) withdifferent lenses depending on the test requirements.

The preferably high-quality cameras, which are used in the preferredsystem, are preferably fitted with a control unit that allows automaticwhite balancing, colour balancing and exposure. These values arepreferably adjustable by hand. This control unit allows the imagequality to be optimally adapted to the test conditions. This ensure avery high image quality for further analysis. There is also an option tozoom in on the image section digitally. Other setting options generallyonly have a limited influence on the generated image.

The movement of the pupils of the test subject is recorded by a secondcamera (77), which is also rigidly connected to the head (80) and whichis directed at one of the two eyes (33) of the test person. The secondcamera (77) can, for example, be a B&W CCD camera and can record the eyemovements of the right eye. The pupil position by the second camera (77)is recorded directly by the eye tracking systems show in the figures,wherein the second camera (77) is pointed directly at the eye (33) ofthe test person. The pupil position can also be recorded via opticalrefraction systems such as mirrors or glass fibre cables, wherein theimage of the eye (33) is refracted to the second camera (77).

Both cameras (76, 77) are attached, for example, to a helmet or a pairof glasses or a similar support that is easy to put on and remove, whichis rigidly connected with the head (80) of the test person. As explainedabove, rigidly connected means that the support and both cameras (76,77) follow all of the movements of the head (80), wherein the freedom ofmovement of the head (80) and the eyes (33) is not restricted in anyway. Attaching the camera (76, 77) to a pair of glasses as an easysupport to put on and remove with direct recording on a mobile recordingdevice allows a particularly high level of mobility of the test subjectand allows a much higher range of testing than with standard systems.

Naturally it is also possible to fit several second cameras (77), forexample, to record both pupils of the test subject. Several firstcameras (76) can also be fitted to record the full visual field of thetest subject, if the focal distance of an individual first camera (76)is not sufficient for this. This allows individual image sequences to berecorded and, as described below, to be evaluated and analysed. The termglance sequence refers here to the sum of the recorded and analysedglances.

Using both cameras (76, 77) provides two video signals that are referredto in the following as eye video and visual field video and representedschematically in FIGS. 22 and 23, which are recorded on a video system.The term video system covers all set-ups that are suitable for recordingfilm data. Analogue film materials such as video tapes or digitalstorage media such as DVDs or similar can also be used. The storage ofindividual images in the memory of a computer is considered recordingwithin the meaning of this invention. Different analogue or digital filmformats can be used such as DV, AVI or MPEG2. When using CCD cameras,all image information is preferably recorded on a digital video system,for example on two mini DV recorders.

In the preferred embodiment, the cameras 76, 77) and the video systemare a hard-wired connection or via a radio link. This enables thewireless transmission of the video signals to the video system.Consequently, this does not restrict the test subject's movement whenwalking, cycling or working, e.g. on scaffolding or construction sites.

It is important that both video signals are synchronised, i.e. so thatthe corresponding individual image of the visual field video (9) can befound and vice versa for each individual image of the eye video.Synchronisation can be carried out with a periodic signal generator andtime code. The recording method is preferably synchronised with a tonepulse, which is recorded on the respective audio tracks. This methodenables other external devices, such as UDS data recorders, GPS systems,etc. to be synchronised in order to synchronise other technical andmedical variables such as the current geographical position or alsoheart or pulse frequency, skin resistance, breathing frequency, etc. ofthe test subject directly with the eye glance behaviour. Synchronisationis important for the subsequent processing or evaluation of both videosignals according to the invention.

The preferred method determines the precise coordinates (xa,ya) of thepupil centre point in the eye video using an image detection programme.This determines the pupil coordinates (xa,ya) for each individual imageof the eye video. The pupil coordinates (xa,ya) in an individual imageof the eye video are sketched in FIG. 22. The pupil coordinates (xa,ya)are preferably determined automatically with an image detectionprogramme. It records the contrasts of the pupils with the surroundingsfor each individual image of the eye video and searches for all of thepoints of the individual image, which are darker than a predetermineddegree of darkness. With these points it is possible to record anddelimit a dark area and ultimately determine the focus of this dark areaautomatically. As the dark area corresponds to the pupils of the testsubject, the focus of the dark area represents the centre of the pupils.The image detection system preferably offers setting options for thecorresponding contrasts and the degree of darkness so that aparticularly high degree of accuracy can be achieved for all individualimages. As already mentioned above, it can be defined that apredetermined number of points on the edge of the pupil are selected,which can be easily and reliably identified due to their contrast withthe surroundings, and that these points are accepted as part of anellipsis, wherein the focus or centre of an ellipsis is calculatedaround the circumference of which the predetermined number of pointslie. The best contrast in the form of a grayscale depth can thus beguaranteed for each individual image under different exposureconditions, which improves the reliability of the pupil coordinates(xa,ya). The grayscale depth is any value, which lies in digital formatbetween 1 and 256, for example, and the percentage share of black andwhite defined at an image point. The highest possible value correspondsto a completely black point, the lowest value to a completely whitepoint. As the pupils probably never reach the full black value duringthe recording, a value must be defined that corresponds to the realexisting pupil grey, at least for this image. The threshold demarcatesall image points that are lighter than the defined grayscale value, alldarker areas are included in the focus determination. Three parametersallow the threshold definition to be optimised. As the exposureconditions often change considerably during the tests conducted within asequence, this threshold definition is preferably also possibleindividually for each image. All settings can be stored in a fileaccording to the high requirements for each image of the sequence. Themethod according to the invention allows a particularly high level ofaccuracy when assigning the pupil coordinates (xa,ya) to the visualfield. The respective degree of accuracy can be visualised.

To achieve a particularly high degree of accuracy for the calculatedpupil coordinates and thus to achieve a particularly accuratedetermination of the visual attention, a preferred embodiment of theinvention provides for a correction of visual defects, particularly lensrectification, perspective correction, image field correction and/orcorrection of the so-called aberrations, such as spherical aberration,chromatic aberration, dispersion, asymmetry errors (coma), astigmatismof uneven groups (astigmatism), curvature of image field, domed picture,optical distortion, and/or monochromatic imaging errors.

In addition, an infrared filter can be placed in front of the camera toimprove the accuracy of the localisation of the pupil centre. Thisimproves the contrasts in the eye video. The IR filter has twoadvantages: Firstly, the eye (33) is illuminated with infrared LEDs (IRLED), which guarantee good contrasts for the eye camera and for furtherprocessing, even in absolute darkness. The filter allows the lightemitted by the LED on to the camera chip, but all other spectral rangesof light are attenuated according to the filter transmission curve.Secondly, the reflections on the pupils caused by sunlight, which havean extremely negative impact on focussing, primarily exist in the bluespectral range. Here again the filter reduces the reflections on thepupils, which are caused by sunlight.

In a further advantageous embodiment of the preferred method, anadditional manual inspection is carried out following the automaticdetermination of the pupil coordinates (xa,ya). If automatic recognitionfails (for example sudden light reflexes on the eye surface, etc.), aprocessor can manually change the image processing parameters. It isalso possible to directly correct the pupil coordinates (xa,ya).

The pupil coordinates (xa, ya) for each individual image of the eyevideo are obtained, for example, in the form of a Cartesian pair ofvalues. Naturally, other co-ordinate systems such as polar coordinatesand so on can also be used. As both cameras (76, 77) are rigidlyconnected with the head (80) of the test subject, a specific position ofthe pupil or pupil centre in the eye video always corresponds to anaccurately defined point of vision B in the visual field video (9). Theeye video and visual field video (9) can therefore be used to calculatethe point on which the test subject is fixed. When assigning pupilcoordinates (xa,ya) to coordinates (xb,yb) of the corresponding point ofvision B, i.e. of the point on which the test subject fixes, the visualfield video must be used to determine the correction function Kinitially between then two pairs of coordinates (xa,ya) and (xb,yb). Thecorrelation between pupil coordinates (xa,ya) and point of vision B onthe visual field video is carried out using a test series (calibration).In this case the test subject fixes the row according to specificpredefined pass points P. The correlation function K between pupilcoordinates (xa,ya) and coordinates (xb,yb) in the visual field video iscreated on the basis of the data measured here.

In the preferred method, the correlation function K between pupilcoordinates (xa,ya) on the eye video and the coordinates (xb,yb) of thecorresponding point of vision B on the visual field video 9 isdetermined automatically. One or more sample glance sequences of thetest subject are first taken at one or more specific predefined passpoints P. A sample glance sequence is a glance sequence that is takenpurely for calibration purposes and during which the test subject looksat predefined pass points P. For example, a specific pass point P can bemarked on a wall. In order to achieve the best possible contrast, ablack mark on an otherwise white surface can be chosen as pass point P.Pass point P is normally a cross or a light point or similar. The testsubject is instructed to fix on this pass point P, wherein the visualfield and the eye of the test subject are recorded by both cameras (76,77). This allows several pass points P to be defined.

As the point of vision B on the recorded visual field video of thesample glance sequence is determined by the known pass point P, thecorrelation function K between the pupil coordinates (xa,ya) on the eyevideo and the coordinates (xb,yb) of the corresponding point of vision Bcan be determined on the visual field video. This is achieved bydetermining the pupil coordinates (xa,ya) in the eye video according tothe method described above for each individual image in the eye video.The coordinates (xb,yb) of the pass point P in the correspondingindividual image are also determined on the visual field video. This ispreferably carried out using an image detection method and/or a patterndetection method, which determines coordinates (xb,yb) of the pass pointP, which can be uniquely identified by its contrast on the visual fieldvideo. However, it is also possible to determine the coordinates (xb,yb)of the pass point P in the visual field video for each individual imageby hand, for example by clicking the mouse. This enables the sampleglance sequence to be evaluated even if the surrounding conditions arepoor, wherein automatic determination of the coordinates (xb,yb) of passpoint P is not possible, for example due to an irregular background.

As a result, the pupil coordinates (xa,ya) in the individual image ofthe eye video can be assigned the coordinates (xb,yb) of the pass pointP in the corresponding individual image of the visual field video. Thecorresponding coordinates in the eye and visual field videos aredetermined and stored for each individual image of the sample glancesequence. All of the data records obtained are used to correlate byquadratic regression the pupil coordinates (xa,ya) on the eye video andthe coordinates (xb,yb) of the corresponding point of vision B on thevisual field video, wherein other methods such as linear regression orstochastic models can be used for the correlation. This gives you acorrelation function K: (xa,ya)→(xb,yb), which uniquely assigns aspecific set of pupil coordinates (xa,ya) on the eye video to thecorresponding coordinates (xb,yb) of point of vision B in the visualfield video.

For the best possible accuracy of the correlation function K, at least25 different positions of the pass point P should be used. Above 100different pass point positions, the accuracy achieved hardly increasesand consequently it is not logical to increase the number of pass pointpositions above this. Therefore between 25 and 100 pass point positionsshould preferably be used. The determined correlation function K can beused to generate all further video sequences of the same test series,i.e. wherein there are no changes concerning the camera positions on thehead of the test subject. Through the digital correlation of both pairsof coordinates, it is possible to determine non-linear correlations.

After calibrating the eye tracking system, it is possible to determineand analyse individual glance sequences. Once the correlation function Khas been determined, the pupil coordinates (xa,ya) on the eye video foreach individual image are used to extrapolate the coordinates (xb,yb) ofthe corresponding point of vision B of the visual field video.

By combining the eye video and the visual field video (9) in a resultsvideo, technical software can position the calculated point of vision Bas centre points of attention on the visual field video (9). Bydetermining the coordinates (xb,yb) of the point of vision B accordingto the invention, it is possible to depict the centre point of attentionwith great accuracy. The point of vision B can be recorded accurately onthe visual field video (9). The point of vision B is preferablyindicated on the visual field video (9) by a clearly visible mark suchas a cross.

By using a method according to the invention it is possible to determinewhich areas in the surroundings of a test subject is given their visualattention, which areas in the surroundings are actually perceived by thetest subject, and which areas the test subject glances at or scans, butso briefly or so far away that there is no ordered perception by thetest subject. Consequently, because the test subject has glanced at anarea he or she will not have registered any of the contents of thisarea. Areas in which ordered perception takes place are referred tobelow by the term fixation. Areas in which eye movement occurs and inwhich no ordered perception takes place are referred to below by theterm saccade.

FIG. 2 shows a cross-section of a human eye (33), wherein areas ofdifferent acuteness of vision are identified. The most important areahere is the so-called foveal area (34), which merely consists of aclosely adjoining central optical axis, and in which the highestacuteness of vision is possible, and therefore also combines the orderedperception of visual stimulation. Standard definitions for the fovealarea (34) used currently assume an initial viewing angle (41) ofapproximately 1° around the optical axis. As explained elsewhere in thisdocument, the first viewing angle (41) of the foveal area (34) dependsconsiderably on the focus and the surroundings. The foveal area (34) issurrounded by the so-called parafoveal area (35) in which the subjectcan still perceive coarse patterns. The so-called peripheral area (36)surrounding this parafoveal area (35) is only sensitive to movement. Theeye cannot perceive a pattern or an object in this peripheral area (36).

In the method according to the invention, directly following points ofvision (37, 38) are at least tested and compared in a comparison devicein relation to compliance with at least the first fixation criterion(25). The comparison device can be any suitable device. Preference isgiven to devices that include electronic logic modules or so-calledlogic gates, which allow a comparison of input data based on Booleanalgorithms. Particular preference is given to devices that use this typeof electronic logic modules in integrated form, particularly in the formof processors, microprocessors and/or programmable logic controllers.Particular preference is given to comparison devices that areimplemented in a computer.

The comparison device processes so-called visual coordinates, which canbe abbreviated in the following as VCO, and which can be determinedbased on a correlation function described above between a visual fieldimage (79) and an eye image (78), wherein other methods or procedurescan be used to determine these VCO. FIG. 1 with the reference sign 2gives a list of possible VCO for individual visual field images, whereFrm is an abbreviation for frame, as Cartesian coordinates.

The first fixation criterion (25) can be any type of criterion, whichallows a differentiation between fixations and saccades. The preferredembodiment of the method according to the invention provides that thefirst fixation criterion (25) is a predefinable first distance (39)around the first point of vision (37), that the first relative distance(40) between the first point of vision (37) and the second point ofvision (38) is determined, and that if the first relative distance (40)is less than the first distance (39), the first and second points ofvision (37, 38) are assigned to the first fixation (48), therefore aslong as a second point of vision (38) following a first point of vision(37) remains within the foveal area (34) of the first point of vision(37) and thus within the area of ordered perception of the first pointof vision (37), ordered perception is not interrupted and thus continuesto fulfil the first fixation criterion (25). This is therefore a firstfixation (48). A particularly preferred embodiment of the methodaccording to the invention provides that the first distance (39) is afirst viewing angle (41), which preferably describes an area (34)assigned to foveal vision, in particular a radius between 0.5° and 1.5°,preferably approximately 1°, and that the distance between the firstpoint of vision (37) and the second point of vision (38) is a firstrelative angle (42). Based on the visual coordinates determined using aneye tracking system, it is possible to determine saccades and fixations(48, 49) simply and accurately. FIG. 6 shows a first fixation (48), forexample, which is formed from a sequence of four points of vision (37,38, 69, 70). FIG. 6 also shows the first distance (39), the firstviewing angle (41), the first relative distance (40) and the firstrelative angle (42). Around each of the four points of vision (37, 38,69, 70) is a first circle (43) with the radius of the first distance(39), wherein it is clearly shown that the following point of vision(38, 69, 70) lies within the first circle (43) with radius firstdistance (39) of the preceding point of vision (37, 38, 69), and thusthe preferred first fixation criteria (25) is met. In order to adapt toobjects that are perceived differently or to different people and/orconditions, a further updated version of the invention provides that thefirst fixation criterion (25), particularly the first distance (39)and/or the first viewing angle (41), can be predefined.

FIG. 7 shows a viewing sequence in which not all points of vision (37,38, 69, 70, 71, 72, 73, 74, 75) satisfy the first fixation criterion(25). The first four points of vision (37, 38, 69, 70) satisfy thefixation criterion (25) and together form the first fixation (48),wherein the following three points of vision (71, 72, 73) do not satisfythe first fixation criterion (25). Only the fourth point of vision (74)following the first fixation (28) satisfies the first fixation criterion(25) compared to the third point of vision (73) following the firstfixation (48). The third point of vision (73) following the firstfixation (48) is therefore the first point of vision (73) of the secondfixation (49), which is formed from a total of three points of vision(73, 74, 75). FIGS. 6 and 7 show illustrative examples, althoughfixations (48, 49) can occur in natural surroundings with a variety ofindividual points of vision. The area between the last point of vision(70) of the first fixation (48) and the first point of vision (73) ofthe second fixation (49) forms a saccade, therefore an area withoutperception. The angle between the last point of vision (70) of the firstfixation (48) and the first point of vision (73) of the second fixation(49) is referred to as the first saccade angle (52).

FIG. 1 shows a block diagram for a method according to the invention,wherein in the first step (1) a visual field video (9) and an eye videoare recorded using an eye tracking system. In a second step (2) the VCOare determined from the visual field video and the eye video, which arecompared in a further step (4) in the comparison device with thedefined, saved, importable or predefinable first fixation criterion(25). The points of vision (37, 38) assigned to a saccade or a fixation(48, 49) can now be output for further evaluation, processing orrepresentation. In particular, it can be provided that the first and thesecond point of vision (37, 38) can be output and marked as the firstfixation (48) or the first saccade.

In the comparison device, two at least directly subsequent visual fieldimages or assigned VCO are compared. By preference it must be providedthat the second visual field image has been recorded after apredefinable first period of time, in particular between 0.005 s and 0.1s, preferably between 0.02 s and 0.04 s, following the first visualfield image. Based on the movement resolution of the human eye (33) inthe foveal area (34), which is only approximately 25 Hz, it is preferredthat the time between two directly following visual field images isapproximately 0.04 s. Depending on the required resolution, furthervisual field images can be recorded and the time between two directlyfollowing visual field images can be reduced, wherein a higher movementresolution is achieved, and/or a predefinable number of visual fieldimages can be skipped, or a lower time resolution can be used forrecording, wherein the movement resolution falls, along with theexpenditure. By comparing the directly following visual field images, itis possible to achieve a high movement resolution as well as a lowsystem complexity, as image selection systems and internal buffers canbe avoided.

FIG. 3 shows a block diagram of a highly preferred embodiment of themethod according to the invention, wherein the method steps according tothe method described above under FIG. 1 provide for subsequentprocessing of the calculated data, therefore whether or not a point ofvision (37, 38) is assigned to a fixation (48, 49) or a saccade. It isprovided that the first relative distance (40) is output together withthe points of vision (37, 38) labelled as the first fixation (48) andthe first saccade respectively. The data is prepared for the firstoutput (10) in a first diagram (11) and/or for second output (5) on avisual field video (9), whereby it is preferred that a visual fieldvideo (9) recorded by the eye tracking system to determine the visualcoordinates of points of vision (37, 38) is output and that at least thepoints of vision (37, 38) for the first fixation (48) or the firstsaccade are depicted in the visual field video (9), wherein it ispossible to evaluate the visual perception quickly and simply.

FIG. 12 shows a screenshot of a preferred user interface (55) of acomputer programme for the execution of a method according to theinvention, wherein at the bottom left the visual field image (9) isdepicted in which, according to the method described below, point ofvision information is output concerning the affiliation of theindividual points of vision (37, 38) to a fixation (48, 49) or asaccade. At the top left of the visual field video (9) a first diagram(11) is output (12) in synchronisation to this, wherein to the right ofthe visual field video (9), also in synchronisation to the image fieldvideo (9), a detailed section of the first diagram (11) is output.Moreover, the preferred user interface has a row of control and/or inputmethods.

In the first diagram (11) the first relative distance (40) between twofollowing points of vision (37, 38) or between two following points ofvision (37, 38), which have been compared in the comparison device withregards to compliance with the first fixation criterion (25), is outputduring an image field video (9). FIG. 8 shows a preferred embodiment ofa first diagram, wherein the time (53) or the sequential number (54) offrames, that is the visual field images, of the visual field video (9)is entered on the x-axis, the first relative distance (40) or the firstrelative angle (42) is entered on the y-axis. Information regardingwhether or not this first relative distance (40) between two followingvisual field images has been assigned to a saccade or a fixation (48,49) is also indicated by the colour design or brightness of theindividual first relative distances (40) or the first relative angle(42) displayed. Based on this type of first diagram (11), it is quickand easy to check visual field videos (9) for perception, particularlyvisual awareness. It can also be provided that in the first diagram (11)a marker is displayed to indicate the point currently represented in thevisual field video (9), wherein the first diagram (11) is continuouslyupdated with continuous visual field video (9) and/or continuouslydisplayed around the fixed marker as a moveable and changing firstdiagram (11).

In addition to outputting the data regarding whether a point of vision(37, 38) is assigned to a fixation (48, 49) or a saccade, in the firstdiagram (11), it can be provided that the corresponding data is outputin a specially adapted visual field video (9), as illustrated in FIG. 3by blocks 6, 7 and 8. Preference is given to three different outputtypes, wherein it can be provided that only one of these output types isoutput, or it can be provided that two or all three output types arerepresented at the same time.

FIG. 9 shows a first preferred output type (6), which is also shown inthe screenshot in accordance with FIG. 12, wherein together with a pointof vision (37) corresponding to the current visual field image displayedin the visual field video (9), a first circle (43) is output uniformlyaround point of vision (37) with the radius of the fits distance (39),and/or together with a point of vision (37) corresponding to the currentvisual field image displayed in visual field video (9), a second circle(44) is output uniformly around the point of vision (37) with the radiusof a predefinable second distance, wherein the second distance ispreferably a second viewing angle, which preferably describes an area(35) assigned to parafoveal vision, particularly with a radius up to 5°and above, wherein when viewing the visual field video (9) the areas canbe identified in which ordered or unordered perception is possible dueto the distribution of the acuteness of vision around the centraloptical axis. In addition, it can be provided that by connectingfollowing points of vision (37, 38) first visual traces (45) aredetermined, which are illustrated at least temporarily in the visualfield video (9), therefore, that the visual traces (45) are hidden fromvisual field video (9), particularly becoming continuously weaker,wherein it is quick and easy to identify which areas of the visual fieldvideo (9) are held in the test subject's memory or short-term memoryduring a short period depending on the person.

A second preferred output type (7) is illustrated in FIG. 10, wherein itis provided that the points of vision (37) corresponding to the firstfixation (48) at least are surrounded uniformly by a third circle (46),wherein the radius of the third circle (46) is a function of thecontinuous duration of the first fixation (48), therefore the thirdcircle becomes increasingly larger as the duration of the respectivefixation continues. In addition, it can be provided that the saccadesbetween two following fixations (48, 49) are connected via the points ofvision by a line. It is preferred that the individual fixations (48, 49)or saccades shown after a predefinable time are hidden again from thevisual field video (9). To ensure that a distinction can be made betweenseveral saccades or fixations (48, 49) that are output at the same time,it can be provided that these are marked with different colours and/orgrayscales, wherein it can be provided that the sequence of fixations isindicated by different colours, grayscales and/or formation of thecircles.

FIG. 11 shows a third preferred output type (8) of the output of thevisual field video (9), wherein it is provided that this is shaded andthe point of vision (37) corresponding at least to the first fixation(48) is shown surrounded uniformly by a fourth circle (47) in principle,wherein the area of the fourth circle (47) is shown lighter, at leasttemporarily, compared to the shaded visual field video (9). Thisrepresents a particularly advantageous design, in the form of aspotlight or a searchlight, as only areas that are output visibly are orhave been actually perceived by the observed. All other areas areshaded, because these have not actually been perceived.

In addition to the output of the visual field video (9) processedaccording to the invention or to the output (12) of the first diagram(11) (FIG. 8) an evaluation of the entire sequence of a predefinablefirst section or the entire visual field video (9) can be provided,wherein a selection (13) (FIG. 3) of a first section of the visual fieldvideo (9) can be provided. In an evaluation unit (14) all of thefollowing points of vision (37, 38, 69, 70) that satisfy the firstfixation criterion (25), together assigned to a first fixation (48), theangular distance between the first point of vision (37) assigned to thefirst fixation (48) and the last point of vision (70) assigned to thefirst fixation (48) is determined and output as the first fixation angle(51) (FIG. 13). In addition, it is preferred that the angular distancebetween the last point of vision (70) assigned to the first fixation(48) and a first point of vision (73) assigned to a second fixation (49)is determined and output as the first saccade angle (52) (FIG. 14). As aresult, it is possible to accurately measure the attention for specificpredefinable objects or scenes of a visual field video (9), as inaddition to the first measured result, therefore whether or not a pointof vision (37) is assigned to a fixation (48) or a saccade, a secondmeasured result is also determined over the duration or local length ofthe fixation (48) or the saccade. It is preferred that for apredefinable first section of the visual field video (9), the frequencyof the determined fixations (48, 49) are output depending on thefixation angle (51), and/or that the frequency of the saccadesdetermined for the first section of the visual field video (9) areoutput depending on the saccade angle (52) or the time. It is preferredthat the fixations (48, 49) satisfy the first fixation criterion (25) orthese are output in a first fixation diagram (15) and that the saccadesdetermined for the first fixation criterion (25) are output in a firstsaccade diagram (20). This enables an entire sequence or a predefinablefirst section to be assessed simply and quickly. FIG. 13 shows such afirst fixation diagram (15), wherein the first fixation angle (51) isentered on the x-axis and the frequency (56) with which the fixations(48, 49) occur with the respective fixation angle (51) are entered onthe y-axis. The first fixation diagram (15) shown in FIG. 13 shows thechanges in fixation during a car journey. FIG. 14 shows a correspondingfirst saccade diagram (20), wherein the first saccade angle (52) isentered on the x-axis and the frequency (56) with which the saccadesoccur with the respective saccade angle (52) are entered on the y-axis.The first saccade diagram (20) shown in FIG. 14 shows the changes insaccade during a car journey. It is preferred that the user interfaceoffers a means for selecting a first section of the visual field video(9).

It can also be provided that the first section in the form of a windowof predefinable size is formed on both sides of the marker shown in thefirst diagram (11) and that the first fixation diagram (15) and/or thefirst saccade diagram (20) generates and displays a constant length forthis first section but a continuously changing content.

In addition or alternatively to the output of the first fixation diagram(15) and/or the first saccade diagram (20), it is preferred that for apredefinable section of the visual field video (9) all of the followingpoints of vision (37, 38, 69, 70), which each satisfy the first fixationcriterion (25), together assigned to a first fixation (48), and that afirst fixation length (103) is determined between the first point ofvision (37) assigned to the first fixation (48) and the last point ofvision (70) assigned to the first fixation (48) and that the frequency(56) of the determined fixations (48, 49) are output depending on thefirst fixation length (103). FIG. 30 shows a preferred output type inthe form of a fixation length diagram (100), wherein the first fixationlength (103) is entered on the x-axis as the duration of a fixation (48,49), wherein the number of frames (106) or the images of a visual fieldvideo (9) can be stipulated as equivalent scaling, and wherein thefrequency (56) with which fixations (48, 49) occur with the respectivefixation length (103) in the predefinable first section of the visualfield video (9) is entered on the y-axis.

Furthermore, it is in particular preferred that for the first section ofthe visual field video (9), a first saccade length (104) between thelast point of vision (70) assigned to the first fixation (48) and afirst point of vision (73) assigned to a second fixation (49), and thatthe frequency (56) of the determined saccades are output depending onthe first saccade length (104). FIG. 31 shows a preferred output type inthe form of a saccade length diagram (101), wherein the first saccadelength (104) is entered on the x-axis as the duration of a saccade,wherein the number of frames (106) or the images of a visual field video(9) can be stipulated as equivalent scaling, and wherein the frequency(56) with which saccades occur with the respective saccade length (104)in the predefinable first section of the visual field video (9) isentered on the y-axis. By outputting the frequency (56) at which thesaccades or fixations (48, 49) occur depending on the first saccadelength (104) or the first fixation length (103), it is possible toquickly and easily analyse that type and quality of attention in thefirst section. This allows object-based and/or situation-baseddifferences to be identified quickly and easily.

In the method according to the invention, periods of time during whichthe test subject's eyes are closed can also be identified automatically.This period of time is triggered by a blink during which the pupil iscovered temporarily by the eyelid. It has been determined that it isuseful to test the first blink length (105) when analysing thephysiological connections as the length of a blink and the frequencywith which blinks of a predefinable first blink length (105) occur. FIG.32 shows a preferred output type as a blink diagram (102), wherein thefrequency (56) at which blinks of a predefinable first blink length(105) occur is output. It has been shown that based on the low frequencyof blinks or the first blink length (105), a high degree of complexityof situation or an object can be concluded and vice versa. In addition,a reduced blink frequency can result in the eye drying out andultimately in eye and/or visual problems. From the above description itcan therefore be concluded that a reduction in visual capacity can beexpected based on the decrease blink frequency.

In addition, in the method according to FIG. 3 there are further optionsfor assessing the determined data, such as other output methods (64, 65,66, 67, 68, 87, 88, 89, 90) which are explained in detail elsewhere.Moreover, it can be provided that the first fixation criterion (25) isreplaced with a second fixation criterion (26) and thereby at least onepredefinable second section of the visual field video (9) is retested todetermine how this is illustrated by the dashed line between theselection (13) of the predefinable first or section of the visual fieldvideo (9) and the first fixation criterion (25).

As illustrated above, and based on a definition of the foveal area (34)as the area in which ordered perception is possible, the first viewingangle (41) of the foveal area (34) depends significantly on the objector surroundings. For example, known objects in a surroundings in whichthe test subject expects this object to appear (such as an octagonalstop sign in road traffic) are very quickly received or detected by thetest subject. Unexpected or unknown objects, on the contrary, are notdetected or perceived as quickly or as uniquely.

Method for measuring the perceptibility of predefinable object units,wherein for a predefinable third section of the visual field video (9)all of the points of vision assigned to a predefinable first object unitare collected in a first object buffer (81) and that the methoddescribed above is carried out with the points of view collected in thefirst object buffer (81). Consequently, at least one object unit isselected for a predefinable or selectable third section of the visualfield video (9), preferably a predefinable number of object units isselected, for example such as in FIGS. 4 and 5 shown with five objectunits. The selection of object units is preferably carried out by auser, wherein however at least one object unit is selectedautomatically. For example, the first object unit can be a stop sign,the second object unit can be a car and the third object unit can be thelane separator on a road. An objective unit within the meaning of thisinvention can also be a scene of the visual field video such astravelling round a bend.

FIG. 4 shows a method in which after selection (13) a third section ofthe visual field video (9), this third section of the visual field video(9) is tested for points of vision, which are assigned or have beenassigned to the predefined object units. Points of vision to be assignedor already assigned as a first object unit refers to all points ofvision that occur between the first point of vision of a first fixationconcerning the first object unit and the last point of vision of a lastfixation concerning the first object unit in the third section of thevisual field video. After the selection (13) of the third section of thevisual field video (9), this is examined (block 91) for aspects that areassigned or have already been assigned to the first object unit. Thisexamination and assignment of individual points of vision to individualobject units can be carried out manually by a user or automaticallyusing a computer, for example with software for the automatic detectionof predefinable optical patterns, such as stop signs, road markings,people and so on.

The points of vision stored in the individual object buffers (81, 82,83, 84, 85) are then, as shown in FIG. 4, processed and analysed usingthe method described above. After the analysis, a fixation diagram (15,16, 17, 18, 19) and a saccade diagram (20, 21, 22, 23, 24) are outputfor each object buffer. Therefore in the preferred method according toFIG. 4 a preferred first fixation diagram (15), a second fixationdiagram (16), a third fixation diagram (17), a fourth fixation diagram(18) and a fifth fixation diagram (19) is output, as well as a firstsaccade diagram (20), a second saccade diagram (21), a third saccadediagram (22), a fourth saccade diagram (23) and a fifth saccade diagram(24). It is thus possible to distinguish and evaluate possible differentobjects in terms of their quality of perception. In particular, it ispossible to assign a so-called request characteristic to variouspossible objects in regard to how strong a persons attention is drawn tothe object in question. Consequently, there are objects that attract theattention of the observer because of their design, wherein other objectsfail to attract the attention of the observer. The understanding of howan object must be designed in order to attract attention or whichobjects attract attention by the observer is important in many areas ofeveryday life, such as in the design of pedestrian crossings, safetyclothing, road layouts or advertising media. In addition, in the methodaccording to FIG. 4 there are further options for assessing thedetermined data, such as other output methods (64, 65, 66, 67, 68, 87,88, 89, 90) which are explained in detail elsewhere.

Known or expected objects are recognised fully as such from the furthestfirst viewing angle (41) before unknown or unexpected objects. As aresult, the acuteness of vision and thus also the foveal area (34) for afirst object or first surroundings can be larger or smaller than for asecond object or second surroundings. The size of the acuteness ofvision required for the specific object therefore represents anextremely meaningful value for the perception of an object or a scenicsequence, wherein the term scenic sequence can relate to allchronologies, such as passing a road or viewing an advertisement.

The larger the area around the central optical axis in or for which anobject is detected, the quicker and easier this is perceived by anobserver and the higher the probability that this is also perceivedcorrectly by the observer as such an object, even if the first fixationcriterion (25) is not satisfied for adjacent objects. For example, anobserver could recognise the advertisements affixed to the roof for aknown soft drinks firm or a known fast-food chain when paying a fleetingglance over a building, whilst the shape of the roof itself is notperceived.

The invention therefore relates to a method for measuring the perceptionof predefinable object units, wherein the method described above iscarried out for at least one predefinable second section of the visualfield video (9) with at least one predefinable second fixation criterion(26) that differs from first fixation criterion (25), wherein thequality of predefinable objects and/or glance sequences can bedetermined in terms of their perceptibility by an observer. FIG. 5 showsa preferred embodiment of such a method as a block diagram, wherein theindividual steps of the method are shown together as a joint dotdashedblock (86). In a preferred embodiment of the invention it is providedthat the second area is identical to the third area, wherein it isparticularly preferred that corresponding methods summarised in block 86apply to the points of vision to be assigned or already assigned to apredefinable first object unit stored or collected in the first objectbuffer (81), as represented in FIG. 5.

In the embodiment according to FIG. 5 it is provided that the secondsection of the visual field video (9) or the content of the first objectbuffer (81), the second object buffer (82), the third object buffer(83), the fourth object buffer (84) and/or the fifth object buffer (85),is processed in series one after the other in the comparison device (4)and the evaluation (14) each with different fixation criteria (25, 26,27, 28, 29), therefore one after the other at least with a firstfixation criterion (25), a second fixation criterion (26), a thirdfixation criterion (27), a fourth fixation criterion (28) and a fifthfixation criterion (29), in the form of a process loop (30) to vary thefixation criterion, wherein the results of a first buffer (31) arestored and then output.

It is preferred that the data determined concerning object perceptiondependent on the respective fixation criterion (25, 26, 27, 28, 29) isoutput. It is preferred that the frequency of fixations (48, 49) isoutput depending at least on the first and the second fixation criterion(25, 26) as the first curve (58) with constant first duration and assecond curve (59) with constant second duration. FIG. 15 shows such asecond diagram, which is referred to as such a fixation level diagram(32), in which the first distance (39) or the first viewing angle (41)is entered on the x-axis and the number (57) of fixation is entered onthe y-axis, and wherein each of the six curves (58, 59, 60, 61, 62, 63)displayed has been determined with different first durations, thereforefor the first curve (58) the distance between the first visual fieldimage (37) and the second visual field image (38) is a frame or a visualfield image, therefore the second visual field image (38) is the visualfield image directly following the first visual field image (37). In thesecond curve (59), the distance between the first visual field image(37) and the second visual field image (38) is two frames. In the thirdcurve (60), the distance between the first visual field image (37) andthe second visual field image (38) is three frames. In the fourth curve(61), the distance between the first visual field image (37) and thesecond visual field image (38) is four frames. In the fifth curve (62),the distance between the first visual field image (37) and the secondvisual field image (38) is five frames. In the sixth curve (63), thedistance between the first visual field image (38) and the second visualfield image (38) is six frames. FIG. 15 shows two different fixationlevel diagrams (32), which each concern different scenes or objects.These fixation level diagrams (32) can be used to quickly determinedifferences that are specific to perception in different objectsdepending on the first distance (39) or the first viewing angle (41) andthe first duration, wherein a scientific evaluation or measurement ofthe different perceptibility of objects is enabled. It is thereforepossible to assign a so-called request characteristic to variouspossible objects in regard to how strong a person's attention is drawnto the object in question.

For further evaluation and analysis of the eye glance behaviour and theobject perception, as already explained above, other output formats canbe provided, as illustrated in FIGS. 16 to 20, and 24 to 27.

FIG. 16 presents all points of vision for a first object unit, thereforeall of the points of vision stored in the first object buffer arerepresented without any special evaluation and/or weighting. Such arepresentation, also referred to as “Dots” (64), allows a practisedobserver to make a series of statements regarding the quality of theobserved object. The greyed out area can, both in its method ofrepresentation as well as in all further methods of representation inaccordance with FIGS. 16 to 20, as well as 24 to 27—also include animage of the first object in the background for ease of understanding,where it must be considered that during dynamic approximation, thepoints of vision represented do not have to be targeted at the areasrepresented in the stored image.

FIG. 18 shows all points of vision preferred for a first object unit,therefore all of the points of vision stored in the first object bufferare represented, wherein all points of vision assigned to a fixation arelabelled, wherein it is preferred that these are represented compared tothe surroundings in a highly perceivable contrast and/or in a highlyperceivable difference in brightness and/or in a different colour to thesurroundings. The points of vision represented and output in this mannerare also referred to as fixed dots (66). As a result, it is possible toevaluate the quality of perception of a first object accurately and ingreater detail. FIG. 18 also represents a first axis system (97), whichmarks the centre point and/or focus of the points of vision.

FIG. 19 also shows all points of vision in the object memory, with allpoints of vision associated with a fixation of predetermined lengthmarked, whereby it is preferred that provision is made to ensure thatthese are shown in easily perceptible contrast against the surroundingsand/or in an easily perceptible brightness difference and/or in a colourthat is different from the surroundings. The points of vision shown ordisplayed thus marked are also called “weighted dots” (67). By changingthe predetermined length of the fixation, it can be quickly and easilyanalysed how the quality of perception of an initial object changesdepending on the length of the individual fixations. FIG. 19 also showsthe first axis intersection (97).

FIG. 20 shows a preferred output form which can be used in addition tothe output forms described elsewhere. Here, a predetermined number ofcircles are marked around the centre (98) of the fixation points of thefirst axis intersection (97). It is preferred that a seventh circle (93)is shown, as illustrated, the diameter of which is formed so that theseventh circle (93) includes fifty percent of the points of vision. Afurther, eighth circle (94) is shown, the diameter of which is formed sothat the seventh circle (93) includes fifty-eight percent of the pointsof vision. A further, ninth circle (95) is shown, so that it includesfifty-nine percent of the points of vision and a tenth circle (96) toshow ninety-nine percent of the points of vision. Thisrepresentation—also called “zone angle” (68)—can be combined with any ofthe other output forms and enables a quick, object-specific evaluationof the quality of perception.

FIG. 17, similarly to FIG. 16, shows the points of vision set out in thefirst object cache, whereby those areas assigned to a fixation after oneof these previous “long” saccades are shown by a sixth circle (92), thesurface of which is shown in easily perceptible contrast against thesurroundings and/or in an easily perceptible brightness differenceand/or in a colour that is different from the surroundings. The centrepoint of the sixth circle (92) is obtained from the centre of the pointsof vision assigned to the relevant fixation. This form of representationis also called the “fixation dominance” (65). The length of a longsaccade is obtained via a predetermined initial saccade angle (52).Alternatively a saccade time period may also be set; a saccade thatexceeds this is a long saccade. The diameter of the sixth circle ispredeterminable and is preferably located in an area that preferablydescribes an area related to parafoveal seeing (35). This representationenables a quick, strong impression to be gained of which areas of anobject attract particular attention from an observer, even in the caseof an unpractised observer. It is also possible to predetermine that thesixth circle (92) will only be shown if the characteristics of fixation(48), (49) and fixation length (103), necessary for the recognition ofan object by the observer, are satisfied. This means it will be seenquickly and clearly whether an object has merely been seen in passing,or has been perceived or actually recognised by an observer.

FIG. 25 shows a tenth preferred output 88, whereby only the saccadesbetween individual fixations are shown, with the last point of vision ofa first fixation being connected by a line with the first point ofvision of a second fixation; the length of the saccade may be shown in adifferent colour, so that an observer can quickly determine the areaswith long perception deficits.

FIG. 24 shows a ninth preferred output (87), in which the field ofvision is overlaid with a grid of predetermined dimensions and/orarrangement, and the individual grid segments (99) are marked withregard to the frequency of the points of vision occurring therein by apredetermined configuration of brightness, colour and/or shading.

FIG. 26 shows an eleventh preferred output, showing the output methodsfrom FIGS. 17, 19, 20, 24 and 25 overlaid over one another, which meansthat a particularly large amount of information can be shown in a singleillustration, and the observer can evaluate an object or an individualscene particularly quickly and easily. FIG. 27 shows a twelfth preferredoutput, where the same view as that of FIG. 26 is shown with an image ofthe first object—in this case an initial scene—behind it, for greaterintelligibility.

In addition to the evaluation and/or output procedures described above,it is particularly recommended to provide a further evaluation and/oroutput procedure as described below, which is especially suited todetermining the complexity of a sequence of scenes. As explained on thebasis of two examples in FIGS. 28 and 29, this is a particularly usefulcombination of already described evaluation and/or output processes,which should ideally be augmented by further beneficial evaluationsand/or outputs.

FIGS. 28 and 29 show a preferred output template (50), with controltools and other output fields omitted leaving only textual descriptions.The two visual field videos (9), are seen adjacent to one another, withone of the visual field videos (9) showing a representation inaccordance with FIG. 17 and the other visual field video (9) showing arepresentation in accordance with FIG. 25. Also, an initial diagram (11)is provided, as well as a detailed view of this initial diagram (11).The output template (50) also includes an initial saccade diagram (20)and an initial fixation duration diagram (100) for the time beingexamined, preferably 2 seconds, but to be predetermined. A seconddiagram (107) is also provided, in which the number of points ofvision—and therefore the frequency of the points of vision—and theirinitial relative distance (40) are arranged around a central visualaxis. It is also preferable that for each currently displayed visualfield video sequence, the initial fixation duration (103), the saccadeangle (52), and a value for the complexity the sequence are given, thevalue for the complexity being determined from the total measuredinitial relative angles (42) measured over a predetermined period,generally one second, and displayed This makes it quick and easy todetermine whether or not a test subject is overwhelmed by a situation.As soon as the value for the complexity exceeds a predetermined limitvalue it may be assumed that well-ordered perception of the objects isno longer taking place. Such a situation in road traffic could havedisastrous consequences. The use of such a procedure as described abovewith this kind of evaluation not only enables a situation to beevaluated, but also enables the quick and easy assessment of whether atest subject is fit to drive.

FIGS. 33 to 36 show examples of a preferred output template 108 using apreferred analysis tool, where FIGS. 33 and 34 form a unit and gotogether and FIGS. 35 and 36 also form a unit and go together. FIGS. 33and 35 each show a visual field video (9), which, in line with the firstpreferred output type 6 of a visual field video (9) with a first andsecond circle (43), (44), is shown in accordance with FIG. 9, wherebythe further preferred output types of a visual field video (9) may alsobe provided. In the visual field video (9) the number of the momentaryvisual field image or frame is presented as a serial number (106), bymeans of which an exact assignment of the visual field image shown at agiven time within the visual field video (9) is possible. Statisticaldata for the current visual field video (9) are also determined,preferably calculated by a computer, and shown in an initial reportstatistics block (109), a second report statistics block (110), as wellas a past statistics block (111) and a future statistics block (112). Ina first and second report statistics block (109), (110) the statisticaldata are shown for any given freely-selectable temporary area of thevisual field video (9). The past statistics block (111) shows thestatistical data for a predetermined time range prior to the momentshown in the visual field image, and the future statistics block (112)shows the statistical data for a predetermined time range after themoment shown in the visual field image.

The individual statistics blocks 109, 110, 111, 112, that is the firstand second report statistics blocks (109), (110), the past statisticsblock (111) and the future statistics block (112), show a complexityvalue, a fixation proportion, a saccade proportion, a fixation factor, asaccade factor and a blink proportion, where MD represents thearithmetical mean, SD the standard deviation or variance, min theminimum, max the maximum and 85% the 85th percentile of the value inquestion for the selected time range of the visual field video (9) ineach statistics block (109, 110, 111, 112).

The complexity here represents the total of all eye movements in theselected time range of the visual field video (9), preferably given indegrees per time unit, e.g. °/s. The fixation proportion represents theproportion of time in the selected time range of the visual field video(9) that can be assigned to fixations, in relation to the whole durationof the selected time range of the visual field video (9); and thesaccade proportion represents the proportion of time in the selectedtime range of the visual field video (9) that can be assigned tosaccades, in relation to the whole duration of the selected time rangeof the visual field video (9). The fixation proportion and the saccadeproportion can each be given values between zero and one and addedtogether give a total of one, as each is only determined using rangesduring which there is no blinking causing a temporary complete darkeningof the eye.

The fixation factor is the ratio of the proportion of fixations to theproportion of saccades at any given time, and the saccade factor is theratio of the proportion of saccades to the proportion of fixations atany given time. The blink proportion is the proportion of time taken upby blinks during the selected time range.

The numbers of fixations, saccades and blinks are mainly shown in therelevant statistics blocks (109, 110, 111, 112) and, as illustrated, arealso shown as discrete values.

FIG. 34 shows the statistical data for the visual field video (9) andthe statistical blocks (109, 110, 111, 112) in accordance with FIG. 33in graphic form, and FIG. 36 shows this statistical data for the visualfield video (9) and the statistical blocks (109, 110, 111, 112) inaccordance with FIG. 35 in graphic form. It is preferable to provide, asillustrated, an initial diagram (11) with the graphic representation ofthe fixations and saccades. The relevant complexity value is also shownin a complexity diagram (113). The further values of the fixationsproportion, saccade proportion, fixation factor and/or saccade factorare furthermore shown in an overview diagram (114). The centrally-placeddouble bar (115) indicates the place shown in the corresponding visualfield video (9). Furthermore, blinks are shown both as a numerical blinkvalue (116) and in the form of a blink bar (117).

The analysis tool in accordance with FIGS. 33 to 36 is especiallyrecommended for the determination and detailed examination of pointswhere there is information loss as a result of high complexity orfrequent foveal, central visual connections. The output of particularlymeaningful statistical values, and their direct assignability to thevisual field video (9) shown, enable qualitative in-depth analyses ofthe real information recording and refined observations of differentinformation deficits and/or information defects to be undertaken. Thusthe degree of visual perception can be determined, enabling furthermedical and neurophysiological investigations to be carried out.

Taking the invention further, provision can be made to evaluate visualperception dimensions together with individual stress parameters (humanphysiological data) and physical movement and condition values, whichmeans that the procedure described above may also be used in a furthercontext of stress and behavioural research.

The invention also relates to a process for monitoring the visualperception of at least one initial, preferably humanoid user, whereby aninitial video of the surroundings of the first user is taken using atleast one initial panoramic camera, so that the first video of thesurroundings is examined on the basis of the presence of at least onesettable pattern, preferably road signs, to determine, using a procedurein accordance with one of Claims 1 to 20, whether the first fixationcriterion (25) is complied with in that the initial pattern providesidentical points of vision at least in places, and that if this fixationcriterion is not fulfilled in the first pattern, at least one control orregulation mechanism is activated. This means that a machine can monitorthe visual range of a user together with their viewing behaviour, and,for instance, determine whether certain predetermined ranges or patternsare or have been perceived by the user. For instance, a car may searchthe street area for road signs, and check whether the driver hasactually perceived the road signs. If this is not the case, the car mayalert the driver by means of an indicator light or sound, or the car maybe automatically stopped if, for example, a stop sign has been missed.

In order to implement a procedure of this kind it is necessary for thepupil movement of the user or the driver to be recorded, for which anappropriate eye tracking system is provided. Although eye trackingsystems firmly linked to the head give the best result, an eye trackingsystem may also be provided that records the pupil movement and line ofvision by means of a number of cameras arranged around the user. Thepreferred application is therefore one in which the user wears gogglesor a helmet in any case, as an eye tracking system of this kind caneasily be integrated into the helmet or goggles. Possible areas ofapplication include fast-moving machines, such as lathes or rope-makingmachines, or helmets for fighter aircraft pilots where the aircraftitself searches the surroundings for targets and risks and the pilot isonly alerted if he has not perceived these. Systems of this kind couldalso be integrated into racing drivers' helmets, and may be optimised bymeans of recognition of the patterns of flag signals at check pointsetc.

Many aspects of the invention can be seen from the description and theillustrations as well as from the Claims, whereby individual features,including in particular those of the different design forms described,could be implemented on a standalone basis or in the form of acombination with at least one other design form of the invention and/orin other areas, with the provision of any combination of features beingpossible, which may represent beneficial, patentable inventions inthemselves. The division of the present application into severalsections does not limit the general validity with regard to theinvention of the statements made within these sections.

1. A method for measuring visual perception, comprising the steps of:processing at least first visual coordinates of a first point of visionassigned to a first field-of-view image, processing at least secondvisual coordinates of a second point of vision assigned to a secondfield-of-view image, with the second field-of-view image being recordedafter the first field-of-view image, examining the second visualcoordinates of the second point of vision together with the first visualcoordinates of the first point of vision in a comparison device andchecking whether they fulfill at least one predetermined first fixationcriterion, assigning the first and second points of vision, providedthey fulfill the at least one first fixation criterion, to a firstfixation assigned to an ordered perception, and marking the first andsecond points of vision as such, and assigning the first and secondpoints of vision, if they do not fulfill the at least one first fixationcriterion, to a first saccade, to be assigned to aleatoric perception,and marking the first and second points of vision as such.
 2. The methodof claim 1, further comprising the step of marking an output of thefirst and second points of vision as belonging to the first fixation orthe first saccade.
 3. The method of claim 1, wherein the first fixationcriterion is a predetermined first distance around the first point ofvision, the method further comprising the steps of determining a firstrelative distance between the first point of vision and the second pointof vision, and assigning the first and second points of vision to thefirst fixation if the first relative distance is less than the firstdistance.
 4. The method of claim 3, wherein the first fixation criterionis predefined.
 5. The method of claim 4, wherein the first distance ispredefined.
 6. The method of claim 4, wherein the predetermined firstdistance is a first angle of view associated with an area assigned tofoveal vision, and wherein the distance between the first point ofvision and the second point of vision is a first relative angle.
 7. Themethod of claim 6, wherein the first angle of view is between 0.5° and1.5°.
 8. The method of claim 6, wherein the first angle of view is about1°.
 9. The method of claim 4, wherein the first relative distance isoutputted together with the points of vision marked as associated withthe first fixation or with the first saccade.
 10. The method of claim 4,wherein the first relative distance is outputted in a first diagram overthe temporal course of a field-of-view video.
 11. The method of claim 1,wherein the second field-of-view image is recorded after a predeterminedperiod of time.
 12. The method of claim 11, wherein the predeterminedperiod of time is between 0.005 and 0.1 seconds.
 13. The method of claim12, wherein the predetermined period of time is between 0.02 and 0.04seconds.
 14. The method of claim 1, wherein the second field-of-viewimage is recorded immediately after the first field-of-view image. 15.The method of claim 1, further comprising the steps of: recording afield-of-view video; determining the visual coordinates of the points ofvision, and displaying in a field-of-view video at least the points ofvision associated with the first fixation or the first saccade.
 16. Themethod of claim 15, wherein a first circle having a radius equal to thefirst distance is displayed substantially uniformly around the point ofvision together with a point of vision corresponding to thefield-of-view image currently shown in the field-of-view video.
 17. Themethod of claim 16, wherein a second circle with a radius equal to apredetermined second distance is displayed substantially uniformlyaround the point of vision together with a point of vision correspondingto the field-of-view image currently shown in the field-of-view video.18. The method of claim 17, wherein the second distance is a secondangle of view describing an area associated with parafoveal vision. 19.The method of claim 18, wherein the second angle of view isapproximately 3° to 5°.
 20. The method of claim 15, further comprisingthe step of connecting sequential points of vision and determining firstview traces, which are shown at least temporarily in the field-of-viewvideo.
 21. The method of claim 15, wherein the points of visionindicated as being associated with at least the first fixation aredisplayed as being substantially uniformly enclosed by a third circle,and wherein the radius of the third circle is a function of thecontinuing duration of the first fixation.
 22. The method of claim 15,wherein the field-of-view video is displayed with shading, wherein thepoints of vision indicated as being associated with at least the firstfixation are displayed as being substantially uniformly enclosed by afourth circle, and wherein the area of the fourth circle is at leasttemporarily displayed with a lighter shading than the shading of thefield-of-view video.
 23. The method of claim 15, wherein for apredetermined first section of the field-of-view video, all sequentialpoints of vision satisfying the first fixation criterion arecollectively assigned to a first fixation, and wherein an angle distancebetween the first point of vision associated with the first fixation andthe last point of vision associated with the first fixation isdetermined and outputted as first fixation angle.
 24. The method ofclaim 23, wherein for the first section of the field-of-view video, anangle distance between the last point of vision associated with thefirst fixation and a first point of vision associated with a secondfixation is determined and outputted as first saccade angle.
 25. Themethod of claim 24, wherein for the first section of the field-of-viewvideo, a frequency of the determined fixations is outputted as afunction of the first fixation angle.
 26. The method of claim 25,wherein for the first section of the field-of-view video, a frequency ofthe determined saccades is outputted as a function of the saccade angle.27. The method of claim 23, further comprising the step of accumulating,for a predetermined third section of the field-of-view video, all pointsof vision associated with a predetermined first object unit in a firstobject cache for measuring perceptibility of the predetermined firstobject unit using the points of vision accumulated in the first objectcache.
 28. The method of claim 1, comprising the steps of: collectivelyassigning all sequential points of vision satisfying the first fixationcriterion for a predetermined first section of the field-of-view videoto a first fixation, and determining a fixation duration between thefirst point of vision associated with the first fixation and the lastpoint of vision associated with the first fixation, and outputting afrequency of the determined fixations are as a function of the firstfixation duration.
 29. The method of claim 1, wherein for the firstsection of the field-of-view video, a first saccade duration between thelast point of vision associated with the first fixation and a firstpoint of vision associated with a second fixation is determined, andwherein a frequency of the determined saccades is outputted as afunction of the first saccade duration.
 30. The method of claim 1,further comprising the step of selecting at least one predeterminedsecond fixation criterion different from the first fixation criterionand performing the method steps of claim 27 for at least onepredetermined second section of the field-of-view video.
 31. The methodof claim 30, wherein a frequency of the fixations as a function of atleast the first and second fixation criteria is outputted as a firstcurve with constant first duration and as a second curve with constantsecond duration.
 32. A method for monitoring the visual perception of atleast one first, preferably humanoid user, comprising the steps of:recording a first video of surroundings of a first user using at leastone first panoramic camera, identifying in the first video at least onepredetermined pattern, preferably a road sign, determining with themethod of claim 1, whether the first fixation criterion is satisfied bypoints of vision that overlap with the at least one predeterminedpattern at least in areas, and activating at least one control circuitif the fixation criterion is not satisfied for the at least onepredetermined pattern.
 33. The method of claim 1, wherein the firstvisual coordinates and the second visual coordinates are determinedusing an eye tracking system.